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RELIABILITY  OF  COMPLEX  DEVICES  IN  RANDOM  ENVIRONMENTS* 

by 

E.  (JINLAR  and  S.  OZEKIci 


Consider  a  device  like  a  jet  engine,  which  is  composed  of  100  to  150 
components.  Each  component  itself  is  made  of  many  parts,  but  for  purposes 
of  repair  and  replacement  the  components  are  indivisible  units.  Reliability 
studies  for  such  a  device  are  made  difficult  by  the  complexity  of  the  device 
and  the  changing  environmental  conditions  it  is  subjected  to. 

The  source  of  difficulty  is  the  stochastic  dependence  of  the  components' 
lifetimes  on  each  other.  This  dependence  is  largely  caused  by  the  fact  that 
the  same  environmental  conditions  affect  all  components  simultaneously. 

Moreover,  the  environmental  conditions  in  the  field  vary  randomly  over 
time  and  are  different  from  the  test  environments  used  in  laboratories.  A 
paper  by  WINTERBOTTOM  (1984)  reviews  some  of  the  recent  developments  on  this 
issue  and  highlights  the  difficulties  involved. 

In  MASTRAN  and  SINGPURWALLA  (1978)  and  SHARED  (1977)  attemps  are  made  to 
confront  the  problem  by  assuming  that  the  lifetimes  are  "associated"  or 
"positively  dependent  by  mixture."  In  LINDLEY  and  SINGPURWALLA  (1984),  for  the 
case  of  two  components,  the  lifetimes  are  assumed  to  be  conditionally  independent 
given  a  certain  parameter,  which  parameter  is  then  randomized.  Thus,  the  random 
parameter  represents  the  effect  of  environment,  which  is  construed  to  be  unchanging 
over  time. 


*Research  was  supported  partially  by  AFOSR  Grant  No.  82-0189  to 
Northwestern  University,  while  both  authors  were  there. 
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Explicit  consideration  of  the  environment  as  a  stochastic  process  has 
been  attempted  in  a  number  of  papers.  Most  such  papers  take  the  model  of 
ESARY ,  MARSHALL,  and  PROSCHAN  (1973)  as  the  starting  point,  and  model  the 
shock  and  wear  "intensities"  as  randomly  varying  in  response  to  environmental 
conditions;  see  A-HAMEED  and  PROSCHAN  (1973),  (1975),  (JINLAR  (1984)  and 
FELDMAN  (1976),  (1977).  These  models  are,  however,  for  one  component,  and 
tend  to  be  statistically  intractable.  The  use  of  multi-components  are 
discussed  by  ESARY  and  MARSHALL  (1974)  ,  and  very  sophisticated  models  are 
introduced  in  (JINLAR  (1977)  .  The  last  paper  models  the  environment  as  an 
abstract  Markov  process,  lets  the  Markov  process  govern  the  intensities  of 
shock  and  wear  for  each  component,  and  view  the  deterioration  levels  of 
components  as  an  n-dimensional  process  with  conditionally  independent  incre¬ 
ments  given  the  intensity  processes.  However,  the  mathematical  complexity 
and  statistical  intractability  of  this  paper  detract  much  from  its  usefulness. 

In  the  present  paper  we  introduce  a  relatively  simple  model.  Environment 
process  is  explicitly  considered,  but  its  effect  on  the  components  is  made 
deterministic.  For  this  purpose,  we  introduce  a  concept,  which  we  call  intrin¬ 
sic  age,  in  order  to  relate  the  deterioration  of  a  component  under  field  condi¬ 
tions  to  the  deterioration  it  would  have  experienced  under  laboratory  conditions. 
The  data  required  are  the  distribution  functions  t  -*■  Fjc(x,t),  one  for  each  com¬ 
ponent  k  and  each  environmental  state  x,  where  F^(x,t)  is  the  probability  that 
the  lifetime  of  the  component  k  is  at  most  t  given  that  the  environmental  state 
remains  fixed  at  x  throughout. 
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We  envision  these  distribution  functions  to  be  obtained,  by  statistical 
and  engineering  considerations,  from  test  data  in  laboratory.  Afterward, 
based  on  these  and  the  probabilistic  structure  of  the  environment  process, 
certain  rules  for  maintenance  and  replacement  decisions  would  be  worked  out 
by  mathematical  means.  For  the  operating  engineer  who  is  to  make  those  deci¬ 
sions  daily,  we  provide  simple  schemes  to  compute  the  quantities  needed  for 
decision  making.  Although  we  shall  develop  the  concept  fully,  it  seems  par¬ 
ticularly  useful  if  the  number  of  environmental  states  can  be  kept  small;  for 
instance,  for  the  jet  engine  alluded  to  above,  it  would  be  helpful  if  the 
environmental  states  were  summarized  as  take-off,  cruise,  landing,  and  idleness, 
instead  of  the  exact  descriptions  provided  by  temperatures,  accelerations,  vibra¬ 
tions,  and  so  on. 

The  paper  is  organized  as  follows.  In  the  first  section,  the  basic  idea 
is  introduced  in  the  simple  case  of  one  component  and  in  a  non-technical  style. 
The  basic  model  is  described  in  the  second  section.  Some  reliability  computa¬ 
tions  are  discussed,  in  the  final  section,  assuming  that  the  environment  evolves 
as  a  semimarkov  process;  these  computations  would  be  needed  to  determine  optimal 
policies  for  maintenance  and  replacement,  which  issues  are  not  touched  in  this 
paper.  Finally,  in  the  appendix  a  brief  account  is  given  of  semimarkov  process¬ 
es  and  related  results. 
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1.  INTRINSIC  AGE  OF  A  COMPONENT 


Our  aim  in  this  section  is  to  introduce  the  basic  idea  in  the  simple 
case  of  one  component  and  in  a  non-technical  style. 

Consider  one  component.  Suppose  that,  throughout  its  lifetime,  the 
environmental  conditions  remain  fixed  at  x.  Let  H(x,t)  be  the  cumulative 
hazard  function  in  t  under  this  condition,  that  is,  if  the  probability  that 
the  component's  lifetime  exceeds  t  is  1  -  F(x,t),  then 


(1.1) 


1  -  F(x,t)  =  exp(-H(x,t))  ,  t  >  0  . 


Observe  that  this  is  also  the  probability  that  L  exceeds  H(x,t)  where  L  is 
an  exponentially  distributed  random  variable  with  mean  one. 

We  view  H(x,t)  as  the  intrinsic  age  of  the  component  at  time  t  given 
that  the  environmental  state  remains  x  throughout  [0,t]  and  that  the  component 

A 

was  new  to  start  with.  Then,  L  may  be  thought  as  the  intrinsic  lifetime; 
when  the  intrinsic  age  reaches  the  intrinsic  lifetime,  the  component  fails. 

Assuming  that  the  distribution  function  t  -*■  F(x,t)  has  a  density,  so  does 
the  increasing  function  t  "*■  H(x,t),  and  the  derivative  of  the  latter  is  called 
the  hazard  rate  function.  In  this  case,  it  is  profitable  to  introduce  a  new 
function,  which  we  call  the  intrinsic  ageing  rate  function,  by 


(1.2) 


r(x,a)  -_H(x,t)|t,T(xa)  ,  a>0, 


where  T(x,a)  is  the  time  at  which  the  intrinsic  age  becomes  a 


(1.3) 


x(x , a)  =  inf  { t :  H(x,t)  >  a}  . 


These  relationships  are  pictured  in  Figure  1  below. 


Figure  1.  Evolution  of  intrinsic  age  under  a  fixed  environ¬ 
mental  state  x.  Here,  £  is  the  intrinsic  lifetime  and  L  is 
the  real  lifetime. 


We  come  now  to  the  main  point.  Suppose  that  the  component  is  subject  to 
varying  environmental  conditions.  Our  basic  assumption  is  that,  at  any  time 
t,  the  rate  (per  unit  time)  of  increase  of  intrinsic  age  is  r(x,a)  if  the 
environment  is  in  state  x  and  the  intrinsic  age  is  a,  this  being  independent 


of  how  the  environment  and  intrinsic  age  have  arrived  at  their  present 


values  x  and  a.  In  other  words,  letting  and  denote  the  environmental 
state  and  intrinsic  age  respectively  at  time  t,  we  assume  that 


(1.4) 


dA^  =  r(Xt,At)  dt  ,  t  >  0  . 


Finally,  as  before,  we  assume  that  the  intrinsic  lifetime  L  is  a  standard 
(mean  one)  exponential  random  variable  independent  of  the  environmental  process 
(Xt) ,  and  we  model  the  lifetime  L  of  the  component  as  the  time  at  which 
intrinsic  age  is  about  to  exceed  L: 


(1.5) 


L  =  inf  (t:  A  >  C) 
t 


By  the  continuity  of  t  At  implicit  in  the  main  assumption,  we  indeed  have 


Next,  we  describe  various  implications  of  the  model  that  (1.4)  and  (1.5) 
sets  up.  We  do  this  in  plain  language,  call  the  results  "principles,"  and 
omit  proofs.  The  first  statement  is  a  re-statement  of  (1.1)  -  (1.2):  if  (A^) 
satisfies  (1.4)  then  r  figuring  in  (1.4)  is  as  described  earlier  by  (1.1)  -  (1.2). 
The  remaining  statements  show  the  workings  of  the  model  in  the  simple  case  where 
the  environmental  process  (Xt)  is  piecewise  constant.  They  suggest  an  easy 
updating  scheme  for  the  intrinsic  age  process,  which  would  be  used  in  practice 
for  making  decisions  on  when  to  do  maintenance. 


(1.6)  IDENTIFICATION  PRINCIPLE.  Suppose  that  the  component  is  new  at  the 
beginning  and  spends  all  its  life  in  a  fixed  environmental  state  x.  Let 

t  "*■  F(x,t)  be  the  distribution  function  of  the  lifetime  under  this  condition. 
Then,  the  intrinsic  age  of  the  component  at  time  t  is 

At  *=  H(x ,  t)  =  -log  [l  -  F(x ,  t)]  . 

(1.7)  AGEING  PRINCIPLE.  Suppose  that  a  mission  is  to  start  at  time  s  and 
last  t  time  units,  and  the  environmental  state  is  to  remain  x  throughout  the 
mission.  If  the  intrinsic  age  at  the  start  is  Ag  =  a,  then  at  the  end  it  is 

Ag+t  =  h(x,a,t)  =  H(x,T(x,a)  +  t)  . 

(See  Figure  2  below.)  We  call  h(x,a,t)  -  a  the  amount  of  ageing  caused  by  the 
mission . 


Figure  2.  If  the  intrinsic  age  is  a  at  the  beginning,  then 
it  is  h(x,a,t)  after  t  time  units  spent  under  condition  x. 
Generally,  t(x,a)  is  not  equal  to  the  starting  time  s. 


As  the  example  shows,  the  ageing  principle  is  in  fact  an  updating 
scheme  assuming  that  the  environmental  state  is  known.  In  practice,  as 
time  flows,  past  history  of  the  environmental  process  will  become  known  and 
the  intrinsic  age  can  be  computed  by  the  principles  here.  The  decisions 
regarding  preventive  maintenance  and  replacement  would  then  be  made  on  the 
basis  of  the  computed  intrinsic  age.  On  the  other  hand,  finding  the  optimal 
maintenance  and/or  replacement  policies  require  computations  of  a  different 
sort:  then  the  future  has  to  be  take  .  into  account,  which  requires  a 
stochastic  description  of  the  environmental  process  (Xt> .  We  shall  take  up 
computations  of  that  kind  in  Section  3. 
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2.  DEVICES  WITH  MANY  COMPONENTS 


In  this  section  we  describe  the  basic  model  for  a  device  with  many 
components . 

Throughout,  (Q,A,P)  will  be  a  complete  probability  space.  We  write 
for  [0,“).  To  save  on  notation,  we  do  not  introduce  O-algebras  explicitly 
on  S^,  and  later,  on  other  spaces. 

Let  E  be  a  set  equipped  with  a  O-algebra.  Elements  of  E  are  called 
environmental  states  (or  states,  simply).  Elements  of  the  a-algebra  on  E  are 
called  the  measurable  subsets  of  E.  We  suppose  that,  for  each  state  x  in  E, 
the  singleton  {x}  is  measurable.  We  distinguish  a  point  in  E,  denote  it  by  6, 
and  think  of  it  as  the  state  that  causes  no  ageing  (idle  state  for  the  device). 
We  let  X  =  (X  )  v  be  a  stochastic  process  with  state  space  E;  it  represents 
the  environmental  process.  For  the  present,  we  leave  its  probability  law 
unspec if ied . 

We  let  K  dene'e  the  collection  of  all  components.  It  is  a  finite  set, 
its  elements  are  called  components.  For  each  k  in  K,  we  let  C(k)  be  an 
exponentially  distributed  random  variable  with  mean  one;  it  stands  for  the 
intrinsic  lifetime  of  the  component  k.  We  assume  that  L(k) ,  keK,  are 

independent  of  each  other  and  of  the  process  X. 

We  let  F  denote  the  collection  of  all  positive  vectors  a  =  (aOO)^^, 
that  is,  F  =  .  Each  a  in  F  represents  a  vector  of  intrinsic  ages  of  all 

components.  We  let  A  =  (At)  _  be  an  increasing  continuous  stochastic  process 

L  it  R  j 

taking  values  in  F;  that  is,  for  each  component  k,  t  -*■  A  ( k)  is  increasing  and 
continuous  and  takes  values  in  E+;  it  represents  the  intrinsic  age  process  for 
component  k. 


We  define  the  lifetime  of  component  k  by 

(2.1)  L(k)  =  inf  {t:  A^(k)  >  C(k) }  , 

and  let  L  =  (L(k)),  be  the  vector  of  lifetimes.  The  following  is  the  main 

kCK 

assumption  regarding  the  structure  of  A. 

(2.2)  HYPOTHESES.  i)  For  each  k£K  there  exists  a  positive  measurable  function 
r^  on  ExF  such  that 

(2.3)  dAt(k)  =  rk(Xt,At)dt  ,  t  >  0  . 

(ii)  We  have  r^(x,a)  strictly  positive  for  each  k£K,  a£F,  and  x£E 
with  x  j  6  .  For  x  =  <5,  the  idle  state  in  E ,  we  have  r^x.a)  =  0  for  all 
k£K  and  a£F. 

The  basic  hypothesis  is  the  first  one:  the  intrinsic  age  process  A  is 
a  deterministic  functional  of  the  environment  process  X.  In  particular,  (2.3) 
implies  that  the  rate  of  increase  of  the  intrinsic  age  of  component  k  at  time 
t  is  a  function  of  the  identity  of  that  component,  the  present  state  of  the 
environment,  and  the  present  intrinsic  ages  of  all  the  components.  Hypothesis 
(ii)  is  a  regularity  condition  that  is  meant  to  ensure  that  (2.3)  has  a  unique 
solution;  in  particular,  it  singles  out  the  state  6  as  the  only  state  that 


causes  no  ageing. 


A  special  case  of  interest  is  when  r^(x,a)  depends  on  a  t  F  =  R+  only 
through  a(k) ,  that  is, 

(2.4)  rk(x,a)  =  ?k(x,a(k))  if  a  ■  (a(k))k£K  ' 

In  this  case,  the  intrinsic  ages  At(k) ,  k£K,  do  not  interact  except  through 
their  common  dependence  on  X.  Then,  the  results  of  the  preceding  section 
applies  to  each  component  separately.  This  case  is  important  in  applications 
because  of  its  computational  simplicity.  But  we  do  not  assume  this  simpler 
case  because  it  does  not  reduce  the  mathematical  development  appreciably. 

In  the  further  special  case  where  rk(x,a)  is  free  of  a,  say, 

(2.5)  r^(x ,a)  =  ?k(x)  , 


we  have 


A^(k)  =  AQ(k)  +  /  ?k(Xs)  ds  , 


0 


kVAs' 


and  t  ■+  At(k)  is  an  additive  functional  of  X.  This  case  has  been  studied  in 
some  depth,  see  PARKUS  and  BARGMAN  (1970)  and  especially  (JINLAR  (1977)  where 
this  case  occurs  as  a  special  case  of  a  model  different  from  the  general  one 
here. 

In  the  special  case,  uninteresting  from  our  present  perspective,  where 
rk(x,a)  is  free  of  x,  the  process  A  becomes  deterministic.  Since  the 
intrinsic  lifetimes  Lfk)  are  independent  of  each  other,  in  this  case  the  life 
times  L(k)  are  also  independent.  Note  that,  though  the  lifetimes  L(k)  are 


■  -  ■  . 
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stochast ically  independent,  this  case  does  accommodate  deterministic  inter¬ 
actions  between  the  intrinsic  age  processes. 

Going  back  to  the  general  case,  we  summarize  our  point  of  view  once 
more.  Every  component  is  endowed  with  an  intrinsic  lifetime  that  is 
independent  of  everything  else  and  has  the  exponential  distribution  with 
mean  one.  The  distinguishing  characteristics  of  a  component  k  are  summarized 
by  the  function  r^(x,a),  which  is  the  response  (the  ageing  rate)  of  k  to  the 
environmental  state  x  at  a  time  when  the  intrinsic  ages  of  the  components  are 
a( j) ,  j£K. 

We  shall  think  of  as  a  function  defined  on  K  and  taking  random  values 

in  I+;  similarly,  we  think  of  elements  a  £  F  as  functions  defined  on  K  and 
taking  values  in  R+.  We  let  r(x,a)  denote  the  function  on  K  whose  value  at 
k  £  K  is  r^(x,a).  For  computational  and  display  purposes,  functions  on  K 
ought  to  be  thought  as  column  vectors.  We  let  c  denote  the  counting  measure 
on  K,  it  can  be  thought  as  a  row  vector  whose  entries  are  all  equal  to  one. 
Thus,  for  a  €  F  we  have 

(2.6)  ca  =  J  a(k)  . 

keu% 
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With  these  notational  conventions,  the  basic  hypothesis  expressed  by 
the  differential  equation  (2.3)  can  be  re-stated  as 


(2.7) 


dA^  =  r(Xt,At)  dt  ,  t  >  0 


Solution  of  this  can  be  obtained  by  mimicking  the  one-dimensional  (one 
component)  case  discussed  in  the  preceding  section.  We  describe  a  few 
details  for  the  special  case  where  the  environment  process  X  is  piecewise 


constant . 


For  fixed  xeE,  let  t  h(x,a,t)  denote  the  solution  of 


(2.8) 


df ( t)  =  r(x,f(t))  dt,  t  >  0  , 


with  the  initial  condition 


(2.9) 


f(0)  =  a  . 


In  other  words,  the  k-entry  of  column  vector  h(x,a,t)  is  the  intrinsic  age 
of  component  k  at  time  s+t  assuming  that  the  environment  remained  in  state  x 
throughout  £s,s+t)  and  the  intrinsic  ages  of  the  components  at  time  s  were 
a(j) ,  jeK.  In  particular,  for  x  =  6,  we  have 


(2.10) 


h(6,a,t)  =  a  ,  t  >  0  , 


because  of  the  assumption  that  r(6,a)  =  0  for  all  a. 
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Next,  we  describe  the  construction  of  the  path  t  A£  as  a  functional 
of  the  trajectory  t  -*•  X  in  the  simple  and  important  case  where  t  -*■  Xt  is 
piecewise  constant.  Let  Tq  ■=  0  and  let  T^,T2»**»  be  the  successive  jump 
times  of  X.  By  the  term  piecewise  constant,  we  mean  that  t  -*■  X  is  constant 
over  each  interval  and  that  the  increasing  sequence  of  times  Tn 

approaches  +00  as  n  goes  to  +  °°.  We  let  Y^  be  the  value  of  for 

t  C  CTn,Tn+l^  ^or  eacb  integer  n  _>  0  ,  that  is, 


(2.11)  X  =  Yn  for  T  <  t  <  T  . 

t  n  n  ~  n+1 

Given  An,  we  first  define  the  values  B  of  A  for  t  =  T  recursively: 
v  n  t  n 


(2.12) 


B0  “ 


Bnfl  =  h(WTn+l  "  Tn) 


n  >  0 


and  then  put 


(2.13)  A  =  h(Y  ,B  , t  -  T  )  for  T  <  t  <  Tn+1  . 

t  n  n  n  n  —  n-rr 

This  defines  At  for  all  t  e  E+,  and  in  view  of  the  way  h  is  defined  as  the 
solution  to  (2.8)  and  (2.9),  the  resulting  path  t  •+■  Af  satisfies  (2.7). 


WwCVv  A  V-  «~-V.  >£/■*  L* 


. 


3.  RELIABILITY  CONSIDERATIONS 


This  section  is  devoted  to  various  computations  on  the  reliability  of 

the  device.  These  computations  would  be  needed  for  purposes  of  decision 

making  on  maintenance  and  replacement  studies. 

Throughout,  vc  assume  that  the  environment  process  X  =  (X  )  is  a 

piecewise  constant  semimarkov  process  with  state  space  E  and  semimarkov 

kernel  Q.  In  other  words,  with  the  jump  times  T^  and  the  successive  states 

Y  defined  as  in  (2.11),  we  assume  that 
n 

(3'1>  P0n+l£C«  Vl  “  Tn  e  DlV-..,Yn;  T0’--"Tn]  =  Q<VC’D) 

for  every  integer  n  j>  0,  measurable  set  C  c  E,  and  Borel  set  D  C  !+. 

For  further  information  on  the  processes  (Xt> ,  (Yn»Tn) »  and  the  semimarkov 
kernel  Q  we  refer  to  the  appendix.  At  this  point,  we  should  mention  that, 
then,  (Yn)  is  a  Markov  chain  with  state  space  E  and  transition  kernel 

(3.2)  P(x,C)  =  Q(x,C,E+)  , 

and  that,  given  the  Markov  chain  (Yr) ,  the  successive  sojourn  durations 
Tj-Tq,T2~T^ , . . .  are  conditionally  independent,  with  the  conditional  distribu¬ 
tion  of  Tn+^  -  Tn  depending  on  Yn  and  Y^^  ondy»  but  otherwise  arbitrary. 

In  the  special  case  where  the  sojourn  distributions  are  exponential  and  depend 
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(3.3)  Q(x,C,D)  =  P(x,C)  J  \(x) c  dt  , 

D 

then  the  semimarkov  process  X  becomes  Markov. 

In  general,  the  sequence  (^n>Tn)  satisfying  (3.1)  is  called  a  Markov 

renewal  process  with  state  space  E  (that  the  Tn  take  values  in  I+  is 

understood).  Moreover,  then,  defining  Bn  =  Aj  as  in  (2.12),  the  sequence 

n 

((Yn,Bn),Tn)  is  again  a  Markov  renewal  process,  the  Markov  chain  (Yn,Bn) 
taking  values  in  the  space  £  =  E  x  F.  In  fact,  in  the  computations  to 
follow,  it  is  (Yn,Bn,Tn)  c^at  plays  the  central  role.  Note  that,  even  if 
we  assumed  E  to  be  discrete,  the  space  E  is  not  discrete  and  hence  there  is 
little  to  be  gained  mathematically  by  specializing  E  to  be  discrete  (computa¬ 
tions  in  practice  are  another  matter).  However,  if  E  is  discrete,  the 
corresponding  formulas  can  be  obtained  by  replacing  dy  and  dz  below  by  y 
and  z  and  changing  the  integrals  over  y  and  z  to  summations. 

Recall  that  the  lifetimes  L(k)  ,  intrinsic  lifetimes  L(k) ,  and  intrinsic 
ages  At(k)  define  random  functions  L,  L,  and  At  on  the  finite  set  K  of 
components.  Therefore,  L  >  At  means  that  L(k)  >  A,(k)  for  all  keK ,  and  L  >  t 
means  that  L(k)  >  t  for  all  keK.  Also,  recall  Lnat  o  is  the  counting  measure 
on  K  and  acts  as  the  summation  operator  as  described  by  (2.6). 

Finally,  we  introduce  the  following  notations  of  convenience.  We  write 
1^  for  the  indicator  function  of  D,  that  is,  lp(x)  is  equal  to  1  or  0  according 
as  x  is  in  D  or  not.  We  let 

(3.4)  H(x,a,t;C)  =  lc(h(x,a,t))  , 
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(3-5)  q(x,t)  •=  Q(x,E,(t,«))  -  P[Tn+1  -  Tn  >  t  |Yn  «  x]  , 

and,  for  all  x£E  and  a£F,  we  write 

(3.6)  pxa[-]  =  p  [*|xo  =  *•  A0  =  a‘  t  >  a]  * 

that  is,  the  conditional  probability  measure  given  that  environment  starts 
in  state  x,  intrinsic  ages  at  a(k) ,  keE,  and  all  of  the  components  are  in 
working  order. 

Reliability  in  a  mission  of  length  t 

By  this  we  mean  the  probability  that  no  component  fails  during  [0,t] 
and  that  there  can  be  no  maintenance,  repair,  or  replacement  during  [0,tj. 

We  compute  this  probability  assuming  that  the  time  0  coincides  with  a  jump 
of  environmental  process  and  that  the  state  x  of  the  environment  and  the 
intrinsic  ages  a(k)  of  the  components  are  known  and  it  is  further  known 
that  all  components  are  in  working  order.  Thus,  what  we  want  is 

(3’7)  PxatL  >  tlm  Pxa  [C  >  AJ  =  Exa  «*p[-a(At  -  a)]  , 

in  view  of  (2.1)  and  the  memorylessness  of  the  exponential  variables  C(k) 
and  their  independence  from  each  other  and  from  X  and  A. 

To  this  end  we  use  a  renewal  argument  at  the  time  T^  of  first  jump  for 
X.  If  T^  >  t,  then  At  =  h(x,a,t)  =  b  and  the  probability  that  no  failures 
occur  is  exp[-o(b-a)]  .  If  T^  =  s  <  t,  then  Ag  =  h(x,a,s)  =  b,  probability 


19- 


that  no  failures  occur  during  [0,s]  is  exp[-a(b-a},  and  the  future  after  s 
is  a  probabilistic  replica  of  that  at  time  0  except  that  the  initial  variables 
now  are  y  and  b,  where  y  is  the  state  that  X  jumps  to  at  time  s  =  T-^.  Thus, 

(3.8)  P  [L  >  t]  =  q(x , t)  exp[-a(h(x ,a,t)  -  a)] 

+  /  Q(x  ,dy  ,ds)  H(x,a,s;  db)  e  3^p  ,  [L>t-s]  . 

E  XF  x  [0 ,  tj  y 

We  introduce  a  semimarkov  kernel  <$  on  the  space  E*F  by,  in  differential  form, 

(3.9)  Q(x,a;  dy,db;  ds)  =  Q(x,dy,ds)  H(x,a,s;  db)  , 

where  we  removed  the  parentheses  in  the  more  correct  notation  Q( (x ,a) ,d (y ,b) ,ds) , 
In  the  notation  of  Appendix,  the  equation  (3.8)  is  a  Markov  renewal  equation: 
letting  f(x,a,t)  denote  the  left  side  and  g(x,a,t)  the  first  term  on  the  right, 
we  have 

(3.10)  f  =  g  +  Q  *  f  . 

According  to  Proposition  (A. 14)  of  Appendix,  (3.10)  has  a  unique  solution, 
f  =  6  *  g,  where  R  is  the  Markov  renewal  kernel  corresponding  to  Q.  Thus, 
the  solution  of  (3.8)  is 

(3.11)  P  [L>t]  =  /  R(x,a;  dy,db;  ds)  q(y,t-s)  exp [-0(h(y ,b ,t-s)  -  b)]  . 

ExF*[0,t] 
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Perhaps  the  more  important  point  here  is  that  (3.8)  has  a  unique  solu¬ 


tion.  For,  computing  6  requires  solving  equations  of  similar  difficulty. 


However,  an  explicit  solution  like  (3.11)  is  valuable  for  theoretical  purposes 


and  is  useful  to  those  who  would  seek  optimal  policies  for  maintenance.  An 


envisioned  use  of  (3.11)  is  mentioned  in  the  next  paragraph. 


Failure  before  maintenance 


At  each  time  Tn  of  change  of  state  for  the  environment,  the  new  state  Yn 


and  the  intrinsic  age  vector  B  =  at  will  be  known  through  computations  of 

11  An 


the  sort  described  in  Section  1.  We  suppose  that  the  times  would  be  the 


only  times  at  which  a  maintenance  action  can  be  taken,  and  the  decision  maker 


would  decide  for  maintenance  by  considering  only  the  state  Y^  and  the  vector 


?n  :  if  Y^  *  y  and  Bn  =  b,  maintenance  would  be  decided  for  if  (y,b)  belongs 


to  a  certain  set  K.  In  other  words,  the  time  of  first  maintenance  would  be 


(3.12) 


T  =  inf  (Tn  :  (Yn,Bn)  e  M}  , 


where  M  is  a  pre-specif ied  measurable  subset  of  ExF.  The  set  M  itself  would 


be  chosen  in  accordance  with  further  studies  considering  costs,  risks,  etc, 


Of  particular  importance  in  such  considerations  would  be  probabilities  like 


PxatL  >  *  which  we  just  examined,  and  P  [l  >  t]  ,  which  we  shall  compute 


next.  For  instance,  if  one  or  the  other  or  some  combination  of  these  two 


numbers  is  "too  low,"  then  the  point  (x,a)  would  belong  to  M. 


*  «  -  4  1 *T »  *  m  w  4  ,  w*K  V, m  -  ^  ■«  4  ^  r  vv  w  * 
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Thus,  the  probability  we  consider  next,  Pxa^L  >  is  the  probability 
that  no  failures  occur  before  the  maintenance  time  T,  given  that  Xq  =  x , 

Aq  =  a,  and  no  failures  occurred  before  the  time  taken  as  0.  We  use  a 
renewal  argument  at  the  time  of  first  jump  for  X,  just  as  in  the  preceding 
computation.  It  yields,  for  (x,a)  t  M, 


(3.13)  Pxa[L>T]  =  /  Q(x,dy,ds)  1  (y,h(x,a,s))  exp  [-o(h(x,a,s)  -  aj\ 

E*R+  11 

+  /  Q(x,dy,ds)  H(x,a,s;  db)  l^Cy ,b) e“°^b_a)  Pyb[L>T]  , 

E><FXR+ 

where  we  put  N  =  (Exf)\M,  the  no-maintenance  set.  This  is  a  linear  equation  of 
the  form 

(3.14)  f(x,a)  =  g(x,a)  +  /  P(x,a;  dy,db)  f(y,b)  , 

ExF 

and  the  sub-Markovian  kernel  P  is  "highly  defective": 

(3.15)  P(x,a;  E.F)  <  /  Q(x,dy,ds)  1  (y,h(x,a,s))  <  Q(x,E,R+)  =  1 

EXR+ 

for  x  4  6,  since  h(x,a,s)  =  b  >  a  for  x  ^  6,  and  for  x  =  6 ,  it  is  clear  that 
the  next  state  will  differ  from  6.  It  follows  that  (3.14)  has  a  unique  solution, 
of  tl  form  f  =  Rg  in  kernel-function  notation,  where  R  is  the  potential  kernel 
corresponding  to  P: 

00 

R(x,a;  dy,db)  =  \  P  (x,a;  dy,db)  . 

n=0 


(3.16) 


Thus,  the  unique  solution  of  (3.13)  is 


(3.17)  PV,[L>T]  -  /  R(x,a;  dy.db)  g(y,b) 


=  /  R(x,a;  dy,db)  /  Q(y,dz,ds) 

ExF  ExI, 


1  (z,h(z,b,s))  exp[-o(h(z,b,s)  -  b)] 


Again,  as  we  remarked  after  (3.11),  this  computation  shifts  the  real  burden  to 
the  computation  of  R.  On  the  latter  issue,  we  should  note  that,  in  view  of 
(3.15),  Pn(x,a;  Exf)  has  a  geometric  tail  in  n,  and  approximating  R  should  be 


At  the  time  of  first  failure  or  maintenance 


Let  T  be  the  time  of  first  maintenance  as  described  earlier,  by  (3.12) 
Let  5  be  T  or  the  time  of  first  failure  (of  some  component),  whichever  is 
smaller : 


(3.18) 


S  =  T  A  inf  L(k) 
kcK 


In  this  paragraph  we  are  interested  in  the  time  S  and  the  states  of  environ¬ 
ment  and  intrinsic  ages  at  the  time  S.  Our  aim  is  to  compute  the  following 
joint  "distributions": 


'-'.vV-'y.- ■  .v  %.  ■  -  v- 
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(3.19)  QM(x,a;  D.B;  t)  =  Pxa[XseD,  AgCB,  S  <  t,  S  =  t] 

(3.20)  Qk(x,a;  D.B;  t)  =  Pxa[xseD,  AseB,  S  <  t  ,  S  =  L(k)]  , 

X/ 

where  D  is  a  measurable  subset  of  E,  B  is  a  Borel  subset  of  F  =  1.  ,  and 
t  e  I+.  Of  course, 

(3.21)  Q  (x.a;  D.B;  t)  +  £  Qk(x,a;  D.B;  t)  =  P  [XgED.AgCB.S  <  t]  . 

keK 

These  distributions  would  be  needed  in  studies  concerning  optimal  maintenance 
and  replacement  and  in  figuring  out  system  performance  characteristics  in  the 
long  run. 

These  computations  are  similar  to  the  preceding  ones.  Letting  N  =  (E*F)\M, 
we  introduce  the  semimarkov  kernel  Q  by 

(3.22)  Q(x,a;  dy,db;  ds)  =  Q(x,dy,ds)  H(x,a,s;  db)e  ^  l^,(y,b)  . 

Then,  using  a  Markov  renewal  argument  at  T^  as  before,  for  (x,a)cK  , 

(3.23)  Q, (x,a;  D,B;  t)  =  J  Q(x,dy,ds)  J  H(x,a,s;  db)e  c^b  lM(y,b) 

D*[o,t]  B 

+  /  Q(x,a;  dy,db;  ds)  Q  (y,b;  D,B;  t-s) 

ExFx[o,t] 


and 


rk(x,b) 


(3.24)  Q  (x,a;  D,B;  t)  =  f  ds  •  q(x.s)  ln(x)  /  H(x,a,s;  db)e  '  -a' 

k  [0,0  D  B 

+  /  Q(x,a;  dy ,db;  ds)  Q,(y ,b;  D,B;  t-s)  . 

E*F*[0,t] 

For  fixed  D  and  B,  each  of  these  equations  is  a  Markov  renewal  equation  of  the 

form  f  =  g  +  Q  *  f  .  Therefore,  by  (A. 14),  each  has  a  unique  solution  of  the 

form  f  =  R  *  g  ,  where 

00 

(3.25)  R  =  l  Qn 

n=0 

is  the  Markov  renewal  kernel  corresponding  to  Q  and  where  g(x,a,t)  is  the 
first  term  on  the  right  of  (3.23)  or  (3.24).  So, 

(3.26)  Q  (x,a;  D,B;  t)  =  /  R(x,a;  dy,db;  ds)  •  /  Q(y,dz,du) 

ExFx[0,t]  Dx[0,t-s] 

•  f  H ( y , b , u ;  dc)  e~c(c_b)  l^(2,c) 


and 


(3.27)  Qk(z,a;  D,B;  t) 


/  R(x,a;  dy ,db ;  ds) 
EXFX  [0,  t] 


[O, t-s j 


du 


q(y.u)  iDCy) 


•  /  H(y,b,u;  dc)  e_0^c  rk(y,c)  . 
B 


APPENDIX 


This  is  a  rapid  account  of  notions  we  need  from  Markov  renewal  theory. 
We  refer  to  CINLAR  (1969)  and  (1975)  for  further  details. 

As  before  we  write  1  for  £o,“)  we  let  E  be  a  measurable  space  (the 
a-algebra  of  measurable  subsets  of  E  will  be  implicit) .  In  applications,  E 
is  sometimes  the  environmental  state  space  E  and  sometimes  the  space 
E  =  E  x  F  . 

A  semimarkov  kernel  on  E  is  a  family  of  numbers  Q(x,C)  in  [0,l3, 
defined  for  each  x  e  E  and  each  measurable  subset  C  of  E  x  JL+,  such  that 

i)  x  ->  Q(x,C)  is  a  measurable  function  on  E,  and 

ii)  C  ->  Q(x,C)  is  a  measure  on  E  x  1  whose  total  mass  is  at  most  one. 

When  C  =  A  x  B  ,  a  measurable  rectangle,  we  write  Q(x,A,B)  instead  of 
Q(x,Ax3).  Similarly,  for  the  measure  element  at  y  e  E  and  t  e  I+  ,  we  write 
Q(x,dy,dt)  instead  of  Q(x,d(y,t)). 

Given  a  semimarkov  kernel  Q  on  E,  its  iterates  Qn  are  defined  recursive¬ 
ly  via 

II  if  x  e  A  and  0  e  B  , 

0  otherwise, 

(A. 2)  Qn+1  (x , A ,B )  =  /  Q(x,dy,dt)  Qn(y,A,B-t)  ,  n  >  0  , 

Exl+ 


-26- 


where  B-t=  {s-t  :  s  £  B}  .  With  these  we  define  a  kernel  R  by 


(A. 3) 


R(x,A,B)  =  l  Qn  (  x  ,  A ,  B  )  , 
n=0 


called  the  Markov  renewal  kernel  corresponding  to  the  kernel  Q,  which  plays 
the  same  role  in  Markov  renewal  theory  as  the  renewal  function  does  in 
renewal  theory. 

Let  Q  be  a  semimarkov  kernel  on  E  and  suppose  that  it  is  not  defective, 


that  is,  Q(x,E,R+)  =  1  for  all  x.  A  Markov  renewal  process  with  semimarkov 


kernel  Q  is  a  discrete  parameter  process  (Y  ,Tn)  taking  values  in  E  x  l+ 
such  that 


(A.4)  p[Yn+1  £  A,  Tn+1  -  Tn  £  B|Y0 . Y^  T„ . T„]  -  Q(Yn,A,B) 


for  all  integers  n  _>  0  ,  all  measurable  ACE,  and  all  Borel  B  C  I+  .  Then, 


(Yn)  is  a  Markov  chain  with  state  space  E  and  transition  kernel 


(A. 5) 


P(x,A)  =  Q(x,A,I.)  . 


Also,  it  is  easy  to  see  that 


(A.  6) 


Qn(x,A,B)  =  p[Yn  e  A,  Td  e  B|Y0  =  x  ,  TQ  =  o]  , 


and  that  R(x,A,B)  is  the  expected  number  of  times  Yn  £  A  and  E  B,  assuming 


that  Yq  =  x  and  Tq  =  0  . 
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Associated  with  a  Markov  renewal  process  (Yn,Tn)  with  Tq  =  0  and  the 
added  condition  that 

(A. 7)  lim  T  =  00  , 

n  n 

we  define  a  continuous  parameter  process  (X^)  taking  values  in  E  by  setting 

(A. 8)  X  =  Y  if  T  <  t  <  T  . 

t  n  n  —  n+1 

Note  that  X  is  piecewise  constant,  its  jumps  occur  at  ,  and  the 

successive  states  it  visits  are  Yq,Y^,...  .  Such  a  process  X  is  called  a 

semimarkov  process  with  state  space  F  and  semimarkov  kernel  Q. 

In  the  applications  here,  we  modeled  the  environment  process  X  as  a 

semimarkov  process  on  the  state  space  E  with  semimarkov  kernel  Q.  In  this 

case,  the  conditions  that  Q(x,E,l.)  =  1  for  all  x  and  that  lim  T  =  +00 

+  n 

are  very  natural.  However,  in  subsequent  computations,  a  number  of  other 
semimarkov  kernels  arose  naturally  but  are  defective:  Q  defined  by  (3.9), 

Q  defined  by  (3.19),  Q,  defined  by  (3.20),  and  Q  defined  by  (3.22)  are  all 

A 

semimarkov  kernels  on  E  =  E  x  F  and  are  all  defective,  that  is,  for  instance, 
Q(x,a;  E,K+)  _<  1  for  most  x.  We  do  not  associate  Markov  renewal  processes 
with  such  defective  kernels. 


Markov  renewal  equations 


Markov  renewal  equations  are  generalizations  of  renewal  equations.  We 
put  here  the  essentials  in  the  general  case  and  give  a  result  that  got  used  a 
number  of  times  in  Section  3. 

Let  Q  be  a  semimarkov  kernel  on  E;  we  do  allow  it  to  be  defective,  that 
is,  Q(x,E,I+)  <  1  is  possible.  For  a  bounded  measurable  function  f  on 
E  x  ,  we  define  the  function  Q  *  f  on  E  x  I+  by 

(A. 9)  Q  *  f(x,t)  =  /  Q(x ,dy ,ds)  f(y,t-s)  , 

Ex[0,t] 

and  define  Q  *  f  and  R  *  f  similarly  (the  latter  is  well-defined  for  f  positive 
but  may  fail  to  exist  for  arbitrary  bounded  f ) . 

Let  f  and  g  be  bounded  positive  measurable  functions  on  E  x  R+  .  Suppose 

that 

(A. 10)  f  =  g  +  Q  *  f  . 

This  is  called  a  Markov  renewal  equation.  The  following  shows  how  to  solve  it 
(theoretically)  and  points  out  its  strong  resemblance  to  renewal  equations. 

(A. 11)  PROPOSITION.  Suppose  that  Qn (x , E ,  [o ,  tj)  0  as  n  —■ >  00  for  all 
x  e  E  and  t  e  E+  .  Let  f  and  g  be  bounded  positive  measurable  functions 
satisfying  (A. 10).  Then, 

(A. 12)  f  =  R  *  g  . 
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PROOF.  We  start  by  observing  that  Qn  *  (Q  *  f)  =  Q  *  (Qn  *  f)  =  Qn+E  *  f  , 
which  follows  from  (A. 9)  and  (A. 2).  Now,  replacing  f  on  the  right  side  of 
(A. 10)  by  g  +  Q  *  f  ,  and  repeating  this  operation,  we  obtain 

f=g+Q*f=(g+Q*g+Q2*g+...+Qn*g)+  Qn+1  *  f 

for  any  integer  n  j>  1  .  In  view  of  (A. 3) ,  the  first  term  on  the  right  increases 
to  R  *  g  ,  and  we  see  that 

(A. 13)  f  =  R  *  g  +  lim  Qn+1  *  f  . 

Fix  x  c  E  and  t  £  !+  ,  and  let  f  be  bounded  by  the  number  b.  Then, 

Qn  *  f (x , t)  =  /  Qn(x,dy,ds)  f (y ,t-s) 

Ex  [°  • t] 

<  b  /  Qn(x,dy,ds)  =  b  Qn (x , E , [0, t] )  , 

Ex[0,t] 

which  approaches  zero  as  n  — ►  00  by  hypothesis.  Thus,  lim  Qn  *  f  =  0  and  in 
view  of  (A. 13)  the  proof  is  complete. 


The  condition  of  the  preceding  proposition,  namely,  that  Qn(x,E,  [o.t])  o 
as  n  ®  ,  is  difficult  to  check  in  practice,  especially  when  Q  is  given  by  a 
complicated  expression  as  Q  and  Q  are.  The  following  simplifies  the  task 
immensely.  We  state  it  in  the  form  needed. 
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(A.14)  PROPOSITION.  Let  Q  be  a  non-defective  semimarkov  kernel  on  E  and 

suppose  that  the  corresponding  Markov  renewal  process  (Y  ,T  )  is  such  that 

n  n 

A  A 

lim  T  =  +  Let  Q  be  a  semimarkov  kernel  on  E  =  E  x  F  such  that 
n 


(A. 15)  Q(x,a;  dy,F;  ds)  £  Q(x,dy,ds) 


for  all  x  e  E,  a  £  F,  y  e  E,  s  e  1+  .  Then, 


(A. 16)  lim  Qn(x,a;  E,F;  [0,t])  =  0  . 


Therefore  if  f  and  g  are  bounded  positive  functions  on  E  x  f  x  satisfying 

/X  A  A 

f  =  g  +  0  *  f  ,  then  f  =  R  *  g  ,  where  R  is  the  Markov  renewal  kernel  cor¬ 
responding  to  Q. 


REMARK.  The  meaning  of  (A. 15)  is  as  follows:  for  any  positive  measurable 
function  f  on  E  x  R+  ,  the  integral  of  f  with  respect  to  the  kernel  on  the  left¬ 
side  is  less  than  or  equal  to  the  integral  of  f  with  respect  to  the  kernel  on 
the  right. 


PROOF.  Suppose  that  we  have  shown  that 


(A. 17) 


Qn(x,a;  E,F;  [o,t])  <  Qn(x,E,  [0,t]) 


Then,  since 


Qn(x,E,[0,t])  =  P[Tn  <  t|Yn  «  x]  , 


and  since  ->  °°  as  n  ->  00  by  hypothesis,  we  see  that 


lim  Qn(x,E,[0,t])  =  0  , 


which  in  turn  implies  (A. 16)  via  (A. 17).  the  last  statement  of  the  proposi 

A 

tion  is  now  a  restatement  of  Proposition  (A. 11),  with  Q  replacing  Q. 

To  complete  the  proof,  then,  we  need  to  show  (A. 17).  We  do  this  by 
induction  on  n.  For  n  =  1  ,  (A. 17)  is  immediate  from  the  condition  (A. 15). 
Suppose  (A. 17)  holds  for  n.  Then, 


Qn+1(x,a;  E,F;  [o.t]) 


=  f  Q  (x  ,a ;  dy.db;  ds)  Q11  (y ,  b ;  E,F;  [o,t-s]) 

ExFx[o,t] 

_<  /  Q(x,a;  dy  ,db ;  ds)  Qn  (y  ,  E  ,  [0 ,  t-s]  )  = 

EXFX  [0 ,  t] 

_<  /  Q(x,a;  d  y ,  F ;  ds)  Qn(y,E,  [0,  t-s])  £ 

E*[0,t] 

<  /  Q(x,dy,ds)  Qn ( y , E , [0 , t ] )  =  Q°+1 (x , E ,  [0 , t] ) 

E*[0,t] 
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Here,  the  first  inequality  is  justified  by  the  induction  hypothesis,  and  the 
last  inequality  by  the  condition  (A. 15).  So,  (A. 17)  holds  for  n  +  1  as  well, 
and  this  completes  the  proof. 

We  remark  that  all  the  conditions  of  the  preceding  proposition  hold 
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